Evaluation and Comparison of Risk Assessment Methods: Predictive Accuracy

The previous research on health risk assessment has provided significant insights into the predictive ability of different methods. However, these studies have some shortcomings in this regard. First, few studies have compared competing models using the same sample of data, making it difficult to assess their relative performance. Second, as a result of an emphasis on searching for an alternative to the AAPCC, most studies have focused on risk assessment for Medicare beneficiaries. Not many have applied these methods to the under-65 population. Finally, a number of the evaluations of predictive accuracy have used risk assessment in either a prospective or retrospective design. Few have explored both approaches using the same data and methods. This study addresses these shortcomings. In particular, we employ a national, standardized database describing use and expenditures for a large nonelderly population. Further, in assessing predictive accuracy, we apply our methods in a uniform way across all competing models to be tested. Finally, we explore both prospective and retrospective risk assessment for each model. In this chapter, we describe the methods and data we used in evaluating predictive accuracy. We also present the findings of our investigation and discuss their implications. As described below, the analysis presented here does not address explicitly risk assessment for high-cost individuals. Instead, we present the methods and results of a separate analysis for these individuals in Chapter IV. This chapter also does not discuss in any detail the general considerations beyond predictive accuracy when comparing different methods, including practicality, administrative feasibility, and incentives for efficiency. Those topics are addressed in Chapter VI. We begin with a discussion of the models we tested.